Evaluating bias in retrieval systems for recall oriented documents retrieval

The evaluation of a retrieval system has always been the focus of research. Most of the retrieval systems seem to be more efficient for precision oriented documents than recall oriented documents since there is a difference between both the recall and precision oriented documents. Therefore, a system that is efficient for the retrieval of precision oriented documents does not need to be good for recall oriented documents as well. Evaluation of retrieval system is very necessary in order to determine whether these methods are suitable for recall oriented documents retrieval or not. We evaluate different retrieval systems for recall oriented documents retrieval. Our main focus is on finding the bias in retrieval systems. We use different retrieval systems for evaluation; in which four are query expansion techniques while the other three retrieve documents without using query expansion techniques. Patent documents are used for analyzing the effectiveness of retrieval systems. Accessibility of documents is measured by retrievability measurement. Lorenz curve and Gini coefficient are used for measuring bias in systems. Our experiments results show that Term Frequency Inverse Document Frequency (TFIDF) is less biased. While exact method show high retrievability inequality. In query expansion techniques language modelling shows less inequality.